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DETAILED ACTION 

In an amendment dated, 8/03/2009, the Applicant amended claim 1; and added 
claims 12-13. Currently claims 1-13 are pending. 

Claim Objections 

1 . Claim 1 is objected to because of the following informalities: 

Lines 3-4 of Claim 1 disclose the phrase "virtual images each of which...". The 
Examiner suggests amending the phrase to read "virtual images A each of which..." 

Line 6 of Claim 1 discloses the phrase "front of operator's eyes,". The Examiner 
suggests amending the phrase to read "front of an operator's eyes,". 

Line 11 of Claim 1 discloses the phrase "in order of each of operation steps...". 
The Examiner suggests amending the phrase to read "in order of a sequence of 
operation steps...". 

Appropriate correction is required. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1-1 1 have been considered but are 
moot in view of the new ground(s) of rejection. 



Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 
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A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claims 1 , 5-6, 9-1 1 are rejected under 35 U.S.C. 1 02(b) as being anticipated by 
Meisner et al (US 6,625,299). 

Regarding claim 1, Meisner et al discloses an operation guiding system 
comprising: 

a virtual image memory configured to store virtual images each of which 
comprises a virtual operation object (Figure 1 and Column 8, lines 28-44 disclose 
having a memory 20 which has a software program 56 which retrieves virtual images 
which are to be integrated with the real images in the field of view of the user.); 

a display unit having a see-through head mounted display positioned in front of 
operator's eyes, the display unit being configured so that the operator can see through 
the see-through head mounted display to directly see a real object itself and can also 
see a virtual image replayed with the see-through head mourned display (Column 7, 
lines 47-67 disclose having a head-mounted display 36 which has an Optical See 
Through display where the virtual object information is presented onto the display and 
the user looks through the display to see the real world object. Figure 6 and Column 8, 
lines 45-57 disclose that a virtual wire 42 is displayed on the Optical See Through 
display 36 while the real wire 44 is seen through the display 36.); 

a virtual image replay means configured to replay the virtual images on said 
display unit in order of each of operation steps (Column 9, lines 13-35 disclose having 
menu selections for showing a previous wire that was worked on, showing the wire that 
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is currently being worked on, showing the next wire to be worked on, and restarting the 
wire harness assembly program.); and 

a virtual image adjusting means configured to adjust the virtual images such that 
a virtual operation object in a virtual image replayed with the display unit will have an 
overlapping relation with the corresponding real operation object itself which the 
operator can directly see through the see-through head mounted display (Column 8, 
lines 45-57 disclose displaying a virtual wire 42 on top of a wiring board 14.). 

Regarding claim 5, Meisner et al discloses the operation guiding system as set 
forth in claim 1 , wherein said virtual image replay means memorizes a correspondence 
between each of the virtual images and each of the operation steps and has a function 
that calls up one of the virtual images corresponding to one of the operation steps 
specified by the operator (Column 9, lines 13-35 of Meisner et al disclose that the 
operator can issue commands to have head-mounted display 36 display a previous wire 
that was worked on, the wire currently being worked on, and a wire that is to be worked 
on next.). 

Regarding claim 6, Meisner et al discloses the operation guiding system as set 
forth in claim 1 , wherein said virtual image replay means memorizes a correspondence 
between each of the virtual images and each of the operation steps, said virtual image 
replay means having a function that replays the virtual images corresponding to the 
operation steps within a range specified by the operator and returns to a first virtual 
image in the range after a replay of the virtual images (Column 9, lines 13-35 of Meisner 
et al disclose that the operator can issue commands to have head-mounted display 36 
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display a previous wire that was worked on, a wire that is to be worked on next, and 
also to restart a wire harness assembly procedure from the beginning.). 

Regarding claim 9, Meisner et al discloses the operation guiding system as set 
forth in claim 1, further including 

an image pickup means configured to take an image of the real operation object 
(Figure 1 of Meisner et al discloses having a camera 26 which takes images of the 
wiring board 14.)and 

a feature point extraction means configured to extract a feature point decided in 
advance with respect to the operation object, from the image taken by said image 
pickup means (Column 7, lines 4-25 and Column 8, lines 1 1 -35 of Meisner et al disclose 
having a tracker application software program 56 which tracks a pattern of fiducials 12 
detected by a camera 26, and determines a position and an orientation of the operator 
with respect to wiring board 14.); 

said virtual image adjusting means changing a position and/or a dimension of 
each of the virtual images displayed on said display unit automatically such that a point 
of the virtual operation object which corresponds to the feature point extracted by said 
feature point extraction means will conform to the position of the feature point (Figure 6 
and Column 8, lines 45-62 of Meisner et al disclose displaying a virtual wire 42 upon 
wire board 14 to designate a desired placement location for a real wire 44.). 

Regarding claim 10, Meisner et al discloses the operation guiding system as set 
forth in claim 1, wherein said virtual image adjusting means includes a manual controller 
by which the operator can manually change a position and/or a dimension of each of the 
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virtual images displayed on said display unit (Column 9, lines 13-35 of Meisner et al 
disclose that the operator can issue commands to control the virtual images that are 
displayed on the head-mounted display 36.). 

Regarding claim 11, Meisner et al discloses the operation guiding system as set 
forth in claim 10, further including a head tracking means configured to track a motion of 
the operator's head (Figure 1 and Column 7, lines 3-46 of Meisner et al disclose 
tracking an orientation and position of the user 24 with respect to the wiring board 14 
using a sensor 26 attached to a headset 38.); 

said virtual image adjusting means correcting the position of each of the virtual 
images displayed on said display unit based on an output of said head tracking means 
(Figure 1 and Column 7, lines 3-46 and Column 8, lines 1 1 -45 of Meisner et al disclose 
adjusting the augmented image displayed to the user according to the position and 
orientation of the headset 38, which contains a head-mounted display 36 and a camera 
26.). 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 2-4 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Meisner et al (US 6,625,299) in view of Jahn et al (US 2002/0089544). 
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Regarding claim 2, Meisner et al discloses the operation guiding system as set 
forth in claim 1. 

Meisner et al fails to teach wherein each of the virtual images includes a line 
drawing outlining the real operation object and a visual information mark for explaining 
the content of each of the operation steps visually. 

Jahn et al discloses wherein each of the virtual images includes a line drawing 
outlining the real operation object and a visual information mark for explaining the 
content of each of the operation steps visually (Figure 3 of Jahn et al discloses that 
virtual images 14 are shown which give information about a portion of the operation 
object.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
that the invention was made to modify the head-mounted display taught by Meisner et al 
with teachings of Jahn et al in order to provide the user of the head-mounted display 
with virtual images that describe what the user is looking at. 

Regarding claim 3, Meisner et al as modified above discloses the operation 
guiding system as set forth in claim 2, further including a visual information mark input 
means which calls up one of the virtual images from said virtual image memory to 
modify and/or add the visual information mark (Figure 3 of Jahn et al discloses that 
virtual images 14 are shown which give information about a portion of the operation 
object.). 

Regarding claim 4, Meisner et al as modified above the operation guiding 
system as set forth in claim 3, wherein each of the virtual images comprises a layer on 
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which the line drawing is drawn and a layer on which the visual information mark is 
drawn (Figure 3 of Jahn et al discloses that virtual images 1 4 are shown with a line 
showing which part of the operation object they correspond to.). 

7. Claims 7-8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Meisner et al (US 6,625,299) in view of Casby et al (US 6,085,428). 

Regarding claim 7, Meisner et al discloses the operation guiding system as set 
forth in claim 1, further including 

said virtual image replay means controlling the replay of the virtual images based 
on the operator's voice (Column 8, lines 63-67 and Column 9, lines 1-12 and lines 13-35 
disclose controlling the virtual images being displayed according to a voice recognition 
system.). 

Meisner et al fails to teach a voice input means for inputting an operator's voice 
and a voice recognition means configured to recognize the voice inputted using said 
voice input means; 

Casby et al discloses a voice input means for inputting an operator's voice and a 
voice recognition means configured to recognize the voice inputted using said voice 
input means (Column 5, lines 17-28 disclose that speech module processor 58 receives 
a voice input and is programmed to allow access to the system based on the voice 
input.); 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
that the invention was made to further modify the head-mounted display system of 
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Meisner et al with the voice control system taught by Casby et al in order to provide a 
hands-free way to control the head-mounted display system. 

Regarding claim 8, Meisner et al as modified above discloses the operation 
guiding system as set forth in claim 1 , further including 

a voice memory configured to store voice data for explaining the content of each 
of the operation steps and a voice output means configured to output the voice data 
(Column 4, lines 8-42 of Casby et al discloses voice messages stored in memory that 
are played to the user based through a speaker 12 describing an operation being 
performed.); 

said virtual image replay means outputting the voice data to said voice output 
means in synchronization with a replay of the virtual images (Column 3, lines 18-21 and 
Column 4, lines 8-42 of Casby et al disclose that visual information is provided to the 
technician in combination with the voice data being provided to the technician.). 

8. Claims 1 2-1 3 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Meisner et al (US 6,625,299) in view of Tanijiri et al (US 2001/0038361 ). 

Regarding claim 12, Meisner et al discloses the operation guiding system as set 
forth in claim 1. 

Meisner et al fails to teach wherein the see-through head mounted display 
comprises a pair of translucent liquid crystal panels. 

Tanijiri et al discloses wherein the see-through head mounted display comprises 
a pair of translucent liquid crystal panels (Figures 6 discloses a see-through image 
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display apparatus with display units 10R and 10L. Figure 2 and Paragraphs [0034] - 
[0036] disclose that each of the display units 10 contains a transmission-type LCD panel 
1 1 , which is translucent.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
that the invention was made that the image display apparatus taught by Tanijiri et al 
could be substituted for the optical see-though head-mounted display device taught by 
Casby et al and provide the user with a headset containing an optical see-through head- 
mounted display. 

Regarding claim 13, Meisner et al as modified above discloses the operation 
guiding system as set forth in claim 1 , wherein the see-through head mounted display 
comprises a pair of prisms each of which has a reflecting surface and a half mirror, and 
a pair of liquid crystal panels of which virtual image enters each prism from one side 
thereof (Figures 2 and 6 disclose a see-through image display apparatus with display 
units 10 which have LCD panels 1 1 which send an image to one end a prism, which has 
a reflecting surface 13 and a hologram element 22, which acts as a half mirror.), 

wherein the reflecting surfaces reflect the virtual image from the liquid crystal 
panels forward (Figure 2 discloses that surface 13 of the prism reflects the image 
received from the LCD panel 1 1 forward.), 

wherein the half mirrors reflect the virtual image backward so that the operator 
can see the virtual image (Figure 2 discloses that hologram element 22 of the prism 
reflects the image received from the LCD panel 1 1 towards the user's eye.). 
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Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to STEPHEN A. BRAY whose telephone number is 
(571)270-7124. The examiner can normally be reached on Monday - Friday, 9:00 a.m. - 
5:00 p.m., EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, AMR AWAD can be reached on (571)272-7764. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/STEPHEN A BRAY/ 
Examiner, Art Unit 2629 

/Amr Awad/ 

Supervisory Patent Examiner, Art Unit 2629 



7 November 2009 



